agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v13 7/8] Row pattern recognition patch (tests). 221+ messages / 2 participants [nested] [flat]
* [PATCH v13 7/8] Row pattern recognition patch (tests). @ 2024-01-22 09:45 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Tatsuo Ishii @ 2024-01-22 09:45 UTC (permalink / raw) --- src/test/regress/expected/rpr.out | 821 +++++++++++++++++++++++++++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/rpr.sql | 392 ++++++++++++++ 3 files changed, 1214 insertions(+), 1 deletion(-) create mode 100644 src/test/regress/expected/rpr.out create mode 100644 src/test/regress/sql/rpr.sql diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out new file mode 100644 index 0000000000..e8998ebf45 --- /dev/null +++ b/src/test/regress/expected/rpr.out @@ -0,0 +1,821 @@ +-- +-- Test for row pattern definition clause +-- +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); +SELECT * FROM stock; + company | tdate | price +----------+------------+------- + company1 | 07-01-2023 | 100 + company1 | 07-02-2023 | 200 + company1 | 07-03-2023 | 150 + company1 | 07-04-2023 | 140 + company1 | 07-05-2023 | 150 + company1 | 07-06-2023 | 90 + company1 | 07-07-2023 | 110 + company1 | 07-08-2023 | 130 + company1 | 07-09-2023 | 120 + company1 | 07-10-2023 | 130 + company2 | 07-01-2023 | 50 + company2 | 07-02-2023 | 2000 + company2 | 07-03-2023 | 1500 + company2 | 07-04-2023 | 1400 + company2 | 07-05-2023 | 1500 + company2 | 07-06-2023 | 60 + company2 | 07-07-2023 | 1100 + company2 | 07-08-2023 | 1300 + company2 | 07-09-2023 | 1200 + company2 | 07-10-2023 | 1300 +(20 rows) + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 150 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 130 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1500 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1300 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | 150 | 90 | 07-06-2023 + company1 | 07-06-2023 | 90 | | | + company1 | 07-07-2023 | 110 | 110 | 120 | 07-08-2023 + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | 1500 | 60 | 07-06-2023 + company2 | 07-06-2023 | 60 | | | + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 07-08-2023 + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test with none greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 0 + company1 | 07-03-2023 | 150 | 3 + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 0 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 0 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 0 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | last_value +----------+------------+-------+------------ + company1 | 07-01-2023 | 100 | 140 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 120 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 1400 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 1200 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | 90 | 120 + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1200 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- match everything +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023 + company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023 + company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023 + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | count +----------+------------+-------+-------------+------------+------- + company1 | 07-01-2023 | 100 | 07-01-2023 | 07-03-2023 | 3 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-06-2023 | 3 + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-09-2023 | 3 + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | 0 + company2 | 07-01-2023 | 50 | 07-01-2023 | 07-03-2023 | 3 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-06-2023 | 3 + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-09-2023 | 3 + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | 0 +(20 rows) + +-- +-- Aggregates +-- +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+-----+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | + company1 | 07-03-2023 | 150 | | | | | | | + company1 | 07-04-2023 | 140 | | | | | | | + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | | | | | | | + company1 | 07-08-2023 | 130 | | | | | | | + company1 | 07-09-2023 | 120 | | | | | | | + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | + company2 | 07-03-2023 | 1500 | | | | | | | + company2 | 07-04-2023 | 1400 | | | | | | | + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | | | | | | | + company2 | 07-08-2023 | 1300 | | | | | | | + company2 | 07-09-2023 | 1200 | | | | | | | + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+------+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + i | v1 | j | v2 +---+----+---+---- + 1 | 10 | 2 | 10 + 1 | 10 | 2 | 11 + 1 | 11 | 2 | 10 + 1 | 11 | 2 | 11 +(4 rows) + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + i | v1 | j | v2 | count +---+----+---+----+------- + 1 | 10 | 2 | 10 | 1 + 1 | 10 | 2 | 11 | 1 + 1 | 10 | 2 | 12 | 0 + 1 | 11 | 2 | 10 | 1 + 1 | 11 | 2 | 11 | 1 + 1 | 11 | 2 | 12 | 0 + 1 | 12 | 2 | 10 | 0 + 1 | 12 | 2 | 11 | 0 + 1 | 12 | 2 | 12 | 0 +(9 rows) + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + tdate | price | first_value | count +------------+-------+-------------+------- + 07-01-2023 | 100 | 07-01-2023 | 4 + 07-02-2023 | 200 | | + 07-03-2023 | 150 | | + 07-04-2023 | 140 | | + 07-05-2023 | 150 | | 0 + 07-06-2023 | 90 | | 0 + 07-07-2023 | 110 | | 0 + 07-01-2023 | 50 | 07-01-2023 | 4 + 07-02-2023 | 2000 | | + 07-03-2023 | 1500 | | + 07-04-2023 | 1400 | | + 07-05-2023 | 1500 | | 0 + 07-06-2023 | 60 | | 0 + 07-07-2023 | 1100 | | 0 +(14 rows) + +-- +-- Error cases +-- +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + ORDER BY tdate + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); +ERROR: syntax error at or near "ORDER" +LINE 6: ORDER BY tdate + ^ +-- pattern variable name must appear in DEFINE +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ END) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: syntax error at or near "END" +LINE 8: PATTERN (START UP+ DOWN+ END) + ^ +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: FRAME must start at current row when row patttern recognition is used +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: SEEK is not supported +LINE 8: SEEK + ^ +HINT: Use INITIAL. diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index f0987ff537..a6542f3dea 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -98,7 +98,7 @@ test: publication subscription # Another group of parallel tests # select_views depends on create_view # ---------- -test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass +test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass rpr # ---------- # Another group of parallel tests (JSON related) diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql new file mode 100644 index 0000000000..0a69cc0e11 --- /dev/null +++ b/src/test/regress/sql/rpr.sql @@ -0,0 +1,392 @@ +-- +-- Test for row pattern definition clause +-- + +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); + +SELECT * FROM stock; + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test with none greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- match everything + +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- +-- Aggregates +-- + +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); + +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- +-- Error cases +-- + +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + ORDER BY tdate + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); + +-- pattern variable name must appear in DEFINE +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ END) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); -- 2.25.1 ----Next_Part(Mon_Jan_22_19_26_18_2024_011)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v13-0008-Allow-to-print-raw-parse-tree.patch" ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-16 23:28 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-16 23:28 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++++++ src/backend/commands/tablecmds.c | 46 ++++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 155 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 81 +++++++++++ 5 files changed, 339 insertions(+), 20 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..8f207c4a79d 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = (Form_pg_constraint) GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = (Var *) linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index e7386e81b07..005fbf78fbf 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8863,6 +8863,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8926,29 +8927,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 75f64628aef..4062f985c5a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,157 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4980,8 +5131,10 @@ alter table testgen.tpart alter column b add generated always as (a * 2) stored; ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" DETAIL: This operation is not supported for partitioned tables. drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects +NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 drop cascades to table testgen.t2 drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index d776595a6ed..821f04e8073 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3197,6 +3197,87 @@ alter table testgen.t3 alter column b add generated always as (a * 2) stored; select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.51.2 --24dzbv6kpqxe4yje-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 66622bf4837..1d98287af62 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8911,6 +8911,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -8974,29 +8975,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2567d918ec3..b3ee82f401e 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4943,6 +4943,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4986,6 +5213,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --tyerjrpxgsfvwown-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 75 +++++++ src/backend/commands/tablecmds.c | 46 +++-- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 232 +++++++++++++++++++++- src/test/regress/sql/alter_table.sql | 147 ++++++++++++++ 5 files changed, 481 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..cfb0a0068cf 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -17,6 +17,7 @@ #include "access/genam.h" #include "access/gist.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/sysattr.h" #include "access/table.h" #include "catalog/catalog.h" @@ -29,6 +30,8 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_relation.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +697,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index aa54c629f8b..c886e6e5ca7 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8938,6 +8938,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, NewColumnValue *newval; RawColumnDefault *rawEnt; Relation pg_attribute; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9001,29 +9002,36 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, /* Make above changes visible */ CommandCounterIncrement(); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); - - /* Make above changes visible */ - CommandCounterIncrement(); - - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); - /* Build a concrete expression for the new default (generated) value */ defval = (Expr *) build_column_default(rel, attnum); defval = expression_planner(defval); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = defval; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + (Node *) defval); + + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 08981f4e380..5346791fd8d 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4955,6 +4955,233 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; t (1 row) +drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; @@ -4998,6 +5225,7 @@ alter table testgen.t3 alter column b ERROR: cannot use subquery in column generation expression drop table testgen.t3; drop schema testgen cascade; -NOTICE: drop cascades to 4 other objects -DETAIL: drop cascades to table testgen.t3 +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table testgen.t5 +drop cascades to function testgen.gen() drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 76187083289..efb08b9ff70 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,153 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) + select x, y + from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + -- tests for invalid invocations alter table doesnotexist alter column foo add generated always as (bar * 2) stored; -- 2.47.0 --4wx636ozsu2eakgd-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
* [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 221+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This builds upon basic support for ... ALTER COLUMN ... ADD GENERATED ALWAYS AS (expr) STORED If we can find a constraint which proves that the given column is already always equal to the new generated column expression, skip the expensive rewrite of the table. The check constraint must use an equality operator which is mergejoinable, and the expression must match exactly the generated column's default expression. --- src/backend/catalog/pg_constraint.c | 73 +++++++++++ src/backend/commands/tablecmds.c | 40 +++--- src/include/catalog/pg_constraint.h | 2 + src/test/regress/expected/alter_table.out | 153 ++++++++++++++++++++++ src/test/regress/sql/alter_table.sql | 83 ++++++++++++ 5 files changed, 335 insertions(+), 16 deletions(-) diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index b12765ae691..37628f65dc1 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "commands/defrem.h" #include "common/int.h" +#include "nodes/nodeFuncs.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -694,6 +695,78 @@ findDomainNotNullConstraint(Oid typid) return retval; } +/* + * Given a relation, an attnum and a (cooked) expression, this returns true if + * it finds a CHECK constraint which proves that the given column is equal to + * the expression. + * + * The constraint must use a mergejoinable operator for the type of the column, + * a concept used by the planner as well to infer equivalence classes on the + * terms in a query (see op_mergejoinable()). + * + * The expressions are compared structurally, so they must match exactly for + * this check to succeed. + */ +bool +findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + bool found = false; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(relid)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (!con->convalidated) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + if (IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var)) && + equal(lsecond(op->args), target_expr)) + { + found = true; + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return found; +} + /* * Given a pg_constraint tuple for a not-null constraint, return the column * number it is for. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 39faef0a114..4afa157ee9e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8939,6 +8939,7 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, Relation pg_attribute; List *newcons; CookedConstraint *cookedDef; + bool rewrite; Assert(def->raw_expr != NULL); Assert(def->cooked_expr == NULL); @@ -9013,25 +9014,32 @@ ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, cookedDef = linitial(newcons); - /* - * Clear all the missing values if we're rewriting the table, since this - * renders them pointless. - */ - RelationClearMissing(rel); + rewrite = !findStructuralCheckConstraintOnAttr(RelationGetRelid(rel), + attnum, + cookedDef->expr); - /* Make above changes visible */ - CommandCounterIncrement(); + if (rewrite) + { + /* + * Clear all the missing values if we're rewriting the table, since + * this renders them pointless. + */ + RelationClearMissing(rel); - /* Drop any pg_statistic entry for the column */ - RemoveStatistics(RelationGetRelid(rel), attnum); + /* Make above changes visible */ + CommandCounterIncrement(); - /* Schedule a rewrite */ - newval = palloc0_object(NewColumnValue); - newval->attnum = attnum; - newval->expr = (Expr *) cookedDef->expr; - newval->is_generated = true; - tab->newvals = lappend(tab->newvals, newval); - tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + } InvokeObjectPostAlterHook(RelationRelationId, RelationGetRelid(rel), attnum); diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h index 1b7fedf1750..7ac9e00c28b 100644 --- a/src/include/catalog/pg_constraint.h +++ b/src/include/catalog/pg_constraint.h @@ -266,6 +266,8 @@ extern char *ChooseConstraintName(const char *name1, const char *name2, extern HeapTuple findNotNullConstraintAttnum(Oid relid, AttrNumber attnum); extern HeapTuple findNotNullConstraint(Oid relid, const char *colname); extern HeapTuple findDomainNotNullConstraint(Oid typid); +extern bool findStructuralCheckConstraintOnAttr(Oid relid, AttrNumber attnum, + const Node *target_expr); extern AttrNumber extractNotNullColumn(HeapTuple constrTup); extern bool AdjustNotNullInheritance(Oid relid, AttrNumber attnum, const char *new_conname, bool is_local, bool is_no_inherit, bool is_notvalid); diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 2d0ce414d12..b269736ab1a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4956,6 +4956,159 @@ select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; (1 row) drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) NOT VALID +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 3) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessShareLock + relation | AccessExclusiveLock +(2 rows) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) + +drop function testgen.gen; +drop table testgen.t5; -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 51d818d4995..9851093fb58 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3198,6 +3198,89 @@ select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; drop table testgen.t3; +-- turning a regular column into a stored generated column +-- without rewriting the table (when a check constraint proves it isn't needed) +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint is not +-- valid +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2) not valid; +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- same as the previous case, but a rewrite happens since the constraint +-- operator is not mergejoinable +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b = a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which skips the rewrite because of the check +begin; +drop trigger testgen_gen on testgen.t5; +alter table testgen.t5 alter column b + add generated always as (a * 2) stored; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +\d+ testgen.t5 +drop function testgen.gen; +drop table testgen.t5; + -- test support for partitioned tables and inheritance create table testgen.tpart (a int, b int) partition by hash (a); create table testgen.tpart_p1 partition of testgen.tpart -- 2.47.0 --zfrmj4necy5zy3mo-- ^ permalink raw reply [nested|flat] 221+ messages in thread
end of thread, other threads:[~2026-05-14 21:51 UTC | newest] Thread overview: 221+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-01-22 09:45 [PATCH v13 7/8] Row pattern recognition patch (tests). Tatsuo Ishii <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:28 [PATCH v1 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 2/2] Try to avoid a rewrite when adding a stored generated column Alberto Piai <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox