agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v25 7/9] Row pattern recognition patch (tests). 184+ messages / 2 participants [nested] [flat]
* [PATCH v25 7/9] Row pattern recognition patch (tests). @ 2024-12-21 06:19 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Tatsuo Ishii @ 2024-12-21 06:19 UTC (permalink / raw) --- src/test/regress/expected/rpr.out | 919 +++++++++++++++++++++++++++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/rpr.sql | 467 +++++++++++++++ 3 files changed, 1387 insertions(+), 1 deletion(-) create mode 100644 src/test/regress/expected/rpr.out create mode 100644 src/test/regress/sql/rpr.sql diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out new file mode 100644 index 0000000000..b8cd6190b4 --- /dev/null +++ b/src/test/regress/expected/rpr.out @@ -0,0 +1,919 @@ +-- +-- Test for row pattern definition clause +-- +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); +SELECT * FROM stock; + company | tdate | price +----------+------------+------- + company1 | 07-01-2023 | 100 + company1 | 07-02-2023 | 200 + company1 | 07-03-2023 | 150 + company1 | 07-04-2023 | 140 + company1 | 07-05-2023 | 150 + company1 | 07-06-2023 | 90 + company1 | 07-07-2023 | 110 + company1 | 07-08-2023 | 130 + company1 | 07-09-2023 | 120 + company1 | 07-10-2023 | 130 + company2 | 07-01-2023 | 50 + company2 | 07-02-2023 | 2000 + company2 | 07-03-2023 | 1500 + company2 | 07-04-2023 | 1400 + company2 | 07-05-2023 | 1500 + company2 | 07-06-2023 | 60 + company2 | 07-07-2023 | 1100 + company2 | 07-08-2023 | 1300 + company2 | 07-09-2023 | 1200 + company2 | 07-10-2023 | 1300 +(20 rows) + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 150 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 130 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1500 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1300 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | 150 | 90 | 07-06-2023 + company1 | 07-06-2023 | 90 | | | + company1 | 07-07-2023 | 110 | 110 | 120 | 07-08-2023 + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | 1500 | 60 | 07-06-2023 + company2 | 07-06-2023 | 60 | | | + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 07-08-2023 + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- basic test with none greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + company | tdate | price | count +----------+------------+-------+------- + company1 | 07-01-2023 | 100 | 0 + company1 | 07-02-2023 | 200 | 0 + company1 | 07-03-2023 | 150 | 3 + company1 | 07-04-2023 | 140 | 0 + company1 | 07-05-2023 | 150 | 0 + company1 | 07-06-2023 | 90 | 0 + company1 | 07-07-2023 | 110 | 0 + company1 | 07-08-2023 | 130 | 0 + company1 | 07-09-2023 | 120 | 0 + company1 | 07-10-2023 | 130 | 0 + company2 | 07-01-2023 | 50 | 0 + company2 | 07-02-2023 | 2000 | 0 + company2 | 07-03-2023 | 1500 | 0 + company2 | 07-04-2023 | 1400 | 0 + company2 | 07-05-2023 | 1500 | 0 + company2 | 07-06-2023 | 60 | 0 + company2 | 07-07-2023 | 1100 | 0 + company2 | 07-08-2023 | 1300 | 0 + company2 | 07-09-2023 | 1200 | 0 + company2 | 07-10-2023 | 1300 | 0 +(20 rows) + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | last_value +----------+------------+-------+------------ + company1 | 07-01-2023 | 100 | 140 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 120 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 1400 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 1200 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | 90 | 120 + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1200 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- match everything +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023 + company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023 + company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023 + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | count +----------+------------+-------+-------------+------------+------- + company1 | 07-01-2023 | 100 | 07-01-2023 | 07-03-2023 | 3 + company1 | 07-02-2023 | 200 | | | 0 + company1 | 07-03-2023 | 150 | | | 0 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-06-2023 | 3 + company1 | 07-05-2023 | 150 | | | 0 + company1 | 07-06-2023 | 90 | | | 0 + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-09-2023 | 3 + company1 | 07-08-2023 | 130 | | | 0 + company1 | 07-09-2023 | 120 | | | 0 + company1 | 07-10-2023 | 130 | | | 0 + company2 | 07-01-2023 | 50 | 07-01-2023 | 07-03-2023 | 3 + company2 | 07-02-2023 | 2000 | | | 0 + company2 | 07-03-2023 | 1500 | | | 0 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-06-2023 | 3 + company2 | 07-05-2023 | 1500 | | | 0 + company2 | 07-06-2023 | 60 | | | 0 + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-09-2023 | 3 + company2 | 07-08-2023 | 1300 | | | 0 + company2 | 07-09-2023 | 1200 | | | 0 + company2 | 07-10-2023 | 1300 | | | 0 +(20 rows) + +-- +-- Aggregates +-- +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+-----+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | | | | | | | 0 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | | | | | | | 0 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | | | | | | | 0 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | | | | | | | 0 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+------+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + i | v1 | j | v2 +---+----+---+---- + 1 | 10 | 2 | 10 + 1 | 10 | 2 | 11 + 1 | 11 | 2 | 10 + 1 | 11 | 2 | 11 +(4 rows) + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + i | v1 | j | v2 | count +---+----+---+----+------- + 1 | 10 | 2 | 10 | 1 + 1 | 10 | 2 | 11 | 1 + 1 | 10 | 2 | 12 | 0 + 1 | 11 | 2 | 10 | 1 + 1 | 11 | 2 | 11 | 1 + 1 | 11 | 2 | 12 | 0 + 1 | 12 | 2 | 10 | 0 + 1 | 12 | 2 | 11 | 0 + 1 | 12 | 2 | 12 | 0 +(9 rows) + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + tdate | price | first_value | count +------------+-------+-------------+------- + 07-01-2023 | 100 | 07-01-2023 | 4 + 07-02-2023 | 200 | | 0 + 07-03-2023 | 150 | | 0 + 07-04-2023 | 140 | | 0 + 07-05-2023 | 150 | | 0 + 07-06-2023 | 90 | | 0 + 07-07-2023 | 110 | | 0 + 07-01-2023 | 50 | 07-01-2023 | 4 + 07-02-2023 | 2000 | | 0 + 07-03-2023 | 1500 | | 0 + 07-04-2023 | 1400 | | 0 + 07-05-2023 | 1500 | | 0 + 07-06-2023 | 60 | | 0 + 07-07-2023 | 1100 | | 0 +(14 rows) + +-- PREV has multiple column reference +CREATE TEMP TABLE rpr1 (id INTEGER, i SERIAL, j INTEGER); +INSERT INTO rpr1(id, j) SELECT 1, g*2 FROM generate_series(1, 10) AS g; +SELECT id, i, j, count(*) OVER w + FROM rpr1 + WINDOW w AS ( + PARTITION BY id + ORDER BY i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (START COND+) + DEFINE + START AS TRUE, + COND AS PREV(i + j + 1) < 10 +); + id | i | j | count +----+----+----+------- + 1 | 1 | 2 | 3 + 1 | 2 | 4 | 0 + 1 | 3 | 6 | 0 + 1 | 4 | 8 | 0 + 1 | 5 | 10 | 0 + 1 | 6 | 12 | 0 + 1 | 7 | 14 | 0 + 1 | 8 | 16 | 0 + 1 | 9 | 18 | 0 + 1 | 10 | 20 | 0 +(10 rows) + +-- Smoke test for larger partitions. +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r+ ) + DEFINE r AS TRUE + ) +) +-- Should be exactly one long match across all rows. +SELECT * FROM s WHERE c > 0; + v | c +---+------ + 1 | 5000 +(1 row) + +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r ) + DEFINE r AS TRUE + ) +) +-- Every row should be its own match. +SELECT count(*) FROM s WHERE c > 0; + count +------- + 5000 +(1 row) + +-- +-- Error cases +-- +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); +ERROR: row pattern definition variable name "up" appears more than once in DEFINE clause +LINE 11: UP AS price > PREV(price), + ^ +-- subqueries in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < (SELECT 100) +); +ERROR: cannot use subquery in DEFINE expression +LINE 11: LOWPRICE AS price < (SELECT 100) + ^ +-- aggregates in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < count(*) +); +ERROR: aggregate functions are not allowed in DEFINE +LINE 11: LOWPRICE AS price < count(*) + ^ +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: FRAME must start at current row when row patttern recognition is used +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: SEEK is not supported +LINE 8: SEEK + ^ +HINT: Use INITIAL. +-- PREV's argument must have at least 1 column reference +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); +ERROR: row pattern navigation operation's argument must include at least one column reference diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index 1edd9e45eb..f5d33b4d7d 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -98,7 +98,7 @@ test: publication subscription # Another group of parallel tests # select_views depends on create_view # ---------- -test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass +test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass rpr # ---------- # Another group of parallel tests (JSON related) diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql new file mode 100644 index 0000000000..a46abe6f0f --- /dev/null +++ b/src/test/regress/sql/rpr.sql @@ -0,0 +1,467 @@ +-- +-- Test for row pattern definition clause +-- + +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); + +SELECT * FROM stock; + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. UP appears twice +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ UP+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test using PREV. Use '*' +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP* DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- basic test with none greedy pattern +SELECT company, tdate, price, count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A A A) + DEFINE + A AS price >= 140 AND price <= 150 +); + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- match everything + +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w, + count(*) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- +-- Aggregates +-- + +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- JOIN case +CREATE TEMP TABLE t1 (i int, v1 int); +CREATE TEMP TABLE t2 (j int, v2 int); +INSERT INTO t1 VALUES(1,10); +INSERT INTO t1 VALUES(1,11); +INSERT INTO t1 VALUES(1,12); +INSERT INTO t2 VALUES(2,10); +INSERT INTO t2 VALUES(2,11); +INSERT INTO t2 VALUES(2,12); + +SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11; + +SELECT *, count(*) OVER w FROM t1, t2 +WINDOW w AS ( + PARTITION BY t1.i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (A) + DEFINE + A AS v1 <= 11 AND v2 <= 11 +); + +-- WITH case +WITH wstock AS ( + SELECT * FROM stock WHERE tdate < '2023-07-08' +) +SELECT tdate, price, +first_value(tdate) OVER w, +count(*) OVER w + FROM wstock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- PREV has multiple column reference +CREATE TEMP TABLE rpr1 (id INTEGER, i SERIAL, j INTEGER); +INSERT INTO rpr1(id, j) SELECT 1, g*2 FROM generate_series(1, 10) AS g; +SELECT id, i, j, count(*) OVER w + FROM rpr1 + WINDOW w AS ( + PARTITION BY id + ORDER BY i + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (START COND+) + DEFINE + START AS TRUE, + COND AS PREV(i + j + 1) < 10 +); + +-- Smoke test for larger partitions. +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r+ ) + DEFINE r AS TRUE + ) +) +-- Should be exactly one long match across all rows. +SELECT * FROM s WHERE c > 0; + +WITH s AS ( + SELECT v, count(*) OVER w AS c + FROM (SELECT generate_series(1, 5000) v) + WINDOW w AS ( + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN ( r ) + DEFINE r AS TRUE + ) +) +-- Every row should be its own match. +SELECT count(*) FROM s WHERE c > 0; + +-- +-- Error cases +-- + +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); + +-- subqueries in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < (SELECT 100) +); + +-- aggregates in DEFINE clause are not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START LOWPRICE) + DEFINE + START AS TRUE, + LOWPRICE AS price < count(*) +); + +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- PREV's argument must have at least 1 column reference +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(1), + DOWN AS price < PREV(1) +); -- 2.25.1 ----Next_Part(Sat_Dec_21_18_20_04_2024_526)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v25-0008-Row-pattern-recognition-patch-typedefs.list.patch" ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 184+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 184+ messages in thread
end of thread, other threads:[~2026-07-15 08:37 UTC | newest] Thread overview: 184+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-12-21 06:19 [PATCH v25 7/9] Row pattern recognition patch (tests). Tatsuo Ishii <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox